The task of the intelligence analyst is an unenviable one. Regardless of whether the intelligence sought is economic, political, military, or gossip-oriented, the task remains the same: deriving useful intelligence data from available sources and collating that data into a meaningful result.
Most analysts (whether they are working for intelligence agencies, the military, or marketing firms, or media) rely on documents, reports, and even stories available from the publicly available media. To this end, intelligence analysts need to read and review hundreds if not thousands of documents. While reading these documents, analysts have to rely on notes, memory, and other means to map out relationships, contexts, and entities mentioned in these documents. Clearly, this is a Herculean task.
It would greatly assist an analyst if entities, concepts, or ideas mentioned together in a document are mapped together. An analyst searching for a specific search term in a multitude of documents would be greatly helped if he or she knew what other terms are linked to that search term in the documents. Normally, as noted above, the intelligence analyst would need to read and digest volumes of documents to obtain the necessary knowledge of the various entities, concepts, or ideas associated with a specific search term in the documents.
To this end, some work has been performed in assisting with the derivation of useful data from documents. Communications between individuals is one of the best sources of information and a study was made in 2004 that analyzed the communications between people within strictly defined confines such as the company Enron (McCallum, A., Corrada-Emmanuel, A., and Wang, X. (2004). The Author-Recipient-Topic Model for Topic and Role Discovery in Social Networks: Experiments with Enron and Academic Email. Technical Report UM-CS-2004-096, 2004.) However, this study did not include an analysis of the content of the communications but merely the author-recipient and topic of the communications.
To date, there does not seem to be any tools available that would assist the analyst in the tasks mentioned above. There is therefore a need for tools that can, preferably, automate some of the tasks mentioned above and hopefully alleviate the workload for analysts.